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We consider catalytic branching populations. They consist of a 
catalyst population evolving according to a critical binary branching 
process in continuous time with a constant branching rate and a 
reactant population with a branching rate proportional to the number 
of catalyst individuals alive. The reactant forms a process in random 
medium. 

We describe asymptotically the genealogy of catalytic branching 
populations coded as the induced forest of R-trees using the many 
individuals — rapid branching continuum limit. The limiting contin- 
uum genealogical forests are then studied in detail from both the 
quenched and annealed points of view. The result is obtained by con- 
structing a contour process and analyzing the appropriately rescaled 
version and its limit. The genealogy of the limiting forest is described 
by a point process. We compare geometric properties and statistics 
of the reactant limit forest with those of the "classical" forest. 

1. Introduction. It is well known that Feller's branching diffusion and, 
more generally, continuous state branching processes appear as weak lim- 
its of a broad range of suitably rescaled Galton- Watson processes (see, e.g., 
[18]). Provided one introduces a suitable topology, the rescaled family forests 
of these Galton-Watson processes converge to limit forests which can be 
represented by paths of a reflected Brownian motion, and more generally 
a reflected Levy process without negative jumps (see [1, 2, 3, 11, 19] and 
references therein). These invariance principles are the main tool for analyz- 
ing the asymptotic behavior of genealogies of Galton-Watson processes. For 
example, one can conclude that the height of a Galton-Watson tree with 
total population size n behaves in distribution, as n — > oo, as 2y/n times the 
height of a standard Brownian excursion. 
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One of the next important steps is to investigate genealogies of multi- 
type branching models with possible interactions between populations. For 
neutral two-type branching models there are three universality classes of 
behavior: independent branching, (one-sided) catalytic branching and mu- 
tually catalytic branching. Loss of independence in the two latter models 
generates many new features and causes great technical difficulties for its 
analysis. 

In the present paper we focus on describing the genealogy of a catalytic 
branching diffusion. This is the many individual fast branching limit of an in- 
teracting branching particle model involving two populations, in which one 
population, the "catalyst," evolves autonomously according to a Galton- 
Watson process, while the other population, the "reactant," evolves accord- 
ing to branching dynamics that are dependent on the number of catalyst 
particles. 

We show that the sequence of suitably rescaled family forests for the re- 
actant population converges in Gromov-Hausdorff topology to a limit forest 
and we characterize the distribution of this limit. We first describe the limit 
of the reactant family forest cut off at a height where the catalyst falls be- 
low a certain threshold. We show that its contour process is given by a path 
of a reflected diffusion process. From that path we derive a collection of 
point processes each describing the mutual genealogical distances between 
all individuals in the population alive at a certain time. 

We start the analysis from a "quenched" point of view, that is, we first 
fix a realization of the catalyst population as an autonomous branching 
process, and consider the reactant population as a branching process in a 
given medium. We then change to an "annealed" point of view and show that 
the joint law of the rescaled catalyst and the reactant population converges 
to a limit. 

Finally, we discuss differences between the reactant limit forest and the 
classical continuum random forest which is known to be associated with 
a reflected Brownian motion. The point processes exhibit the difference in 
genealogical structure between the reactant population family forests and 
the classical Brownian continuum random tree. The differences are due to 
two different effects: the vanishing branching rate once the catalyst becomes 
extinct and the temporal inhomogeneity of the branching rates. The differ- 
ences are roughly as follows. If we pick from the population of the reactant at 
time t two individuals at random, their chance of being from different trees 
is smaller than in the classical case (with a suitable choice of the branching 
rate). Furthermore the total tree length becomes infinite in a stronger sense, 
namely instead of having Hausdorff dimension 2 with finite Hausdorff mea- 
sure function, we now have a Hausdorff measure function that grows faster 
than a square. As a consequence, with positive probability the reactant limit 
forest cannot be associated with the path of a diffusion. 
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Outline. The rest of the paper is structured as follows. In Section 2 we 
provide the underlying spaces and the topologies we will need for represent- 
ing the genealogical forests of our branching model. In Section 3 we define 
our catalytic particle model, its genealogical forests, the associated contour 
processes, the genealogical point processes and the suitably rescaled versions 
of all of these objects. In Section 4 we state our results for the reactant limit 
forest and its differences from the classical Brownian forest. Subsequent sec- 
tions contain proofs of these results. In Section 5 we show that the family 
of suitably rescaled catalyst forests has a limit forest. In Section 6 we prove 
the statement on the limit of the reactant contour processes. In Section 7 
we use the limit contour process to uniquely characterize the distribution 
of the limit forest. In Section 8 we prove results on the limit reactant ge- 
nealogical point process given the catalyst medium. In Section 9 we show 
that the joint law of catalyst and reactant tree converges. Finally Section 10 
contains calculations establishing the differences between the reactant and 
the classical forest. 

2. Preliminaries on trees and their representations. In this section we 
introduce the relevant spaces and topologies for both the particle model and 
its diffusion limit. 

The standard way of representing genealogical relationships between indi- 
viduals in a branching population is with a family forest. The family forest 
consists of as many trees as the initial number of individuals. Each individ- 
ual in a branching process has an edge associated to it whose length is equal 
to its lifetime. When an individual gives birth the edge branches into two 
new edges, while its death turns its edge end into a leaf. In particular, time 
in the branching process corresponds to height in the family forest which is 
measured in terms of the distance to the roots. 

A family forest can be embedded in a plane. A planar embedding implic- 
itly requires a linear (total) ordering to be placed on individuals, providing 
each with a label. A lexicographic labeling, also called the Ulam-Harris la- 
beling, gives each individual a label from the set N", where n S {1,2, . . .} is 
the number of ancestors the individual has had from the start of the process. 
The initial individuals are given labels between 1 and the initial population 
size, distributed in a random order to them. Each parent subsequently gives 
a label to its children that consists of its own label followed by a number 
between 1 and the total number of its children, distributing these randomly 
among its children. See Figure 2 for more detail, and [21] or [11] for a formal 
definition. 

A planar representation and a lexicographic order are the most convenient 
way to visualize a family forest with a linear order. However, the appropriate 
space for family trees in general is the set of all rooted M-trees with a linear 
order on them. 
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Rooted linearly ordered M-trees. A metric space (T, d) is called an M-tree 
if it is path-connected and satisfies the so-called "four point condition," that 
is, d{xi,X2) + d{x3,X4) < max{d{xi,X3) + d{x2,X4),d{xi,X4) + d{x2,X3)}, for 
all xi, . . . , X4 G r. We also refer the reader to [6, 8, 9, 10, 27] for a background 
on M-trees. A rooted M-tree, (T,d,p), is an R-tree {T,d) with a distinguished 
point p €T that we call the root. A forest is just a collection of rooted M- 
trees, and we can always associate a forest of rooted M-trees with an M-tree 
with the root being of degree greater than 1 by gluing together all the roots. 
In our paper we will think of a forest as a rooted M-tree whose root will have 
degree 1 or larger. 

Let T''""* denote the collection of all root-invariant isometry classes of 
rooted compact M-trees, where we define a root-invariant isometry to be an 
isometry i : (Ti , PTi ) — *■ {T2,dT2, PT2) which in addition satisfies i(pTi) = 
PT2 ■ Let T = {i} denote the points of a metric space (T, d, p) G T™°*. For all 
t>0, let dQt denote the set of individuals alive at time t in the branching 
population. Define the genealogical distance metric for {T,d,p) by setting 
for any ti,t2>0 and any ii £ dQt^ , L2 G SQta 



where T(ii, 62) denotes the death of, or splitting time from the "most recent 
common ancestor" of the individuals ii and i2 alive at times ti and t2, 
respectively. Moreover, for any two p, p' G dQo by our convention we have 



For an M-tree (T, d) the root p & T defines a partial order < on T. For 
all y GT, let [p, y] denote the unique closed arc between them, that is, the 
image of the unique geodesic between p and y. We define x <y if x £ [p,y]. 
A linear order (total order) relation on a rooted ordered M-tree is a total 
order which is compatible with the partial order [condition (i)] and fulfills 
the following [condition (ii); see Figure 1]: 

(i) For all X, y G T if X < y, then x y. 

(ii) For all x,y with x y, if x', y' are such that x Ay <x' Ax 
and X Ay <y' Ay then x' <'"^ y'. 



d{ii,i2) := ti+t2- 2r(ti, 62) 



d{p.p') = Q. 




Fig. 1. Illustrates condition (ii)/ ifx<^^'^y, x' is in the same "subtree above" xAy as 
X, and y' is in the same "subtree above" x f\y as y, then x' <''" j/'. 
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Then {T,d,p,< ) is a rooted W-tree with a linear order. Let T''°°''' de- 
note the set of all root-and-linear order invariant isometry classes of com- 
pact M-trees, where we define a root-and-linear root order invariant isom- 
etry z : (T, dr, PT, <r^) — > {T' , dx' , Pt' , ^t^) to be a root invariant isome- 
try i: {T,dT,PT) {T',dT',PT') which in addition satisfies i{u) <)^? i{u') iff 

Remark 2.1. For any element of TP™"*''™ there is a planar representative 
of this isometry class. It is visually convenient to use it as our canonical 
representative when describing a family forest. 

Contour processes. There is a long tradition of coding M-trees via ex- 
cursions (see [3, 11, 19]) which we briefly recall. Write C^+[0,oo) for the 
space of continuous functions e : [0, oo) R+ with e(0) = 0. We associate 
each e G C^+[0,oo) with an R-tree as follows. Define an equivalence rela- 
tion ~e on M+ by letting m ~e U2 iff e{ui) = infu(z[uiAu2,uiVu2] (^(u) = e{u2). 
Consider the following pseudo-metric on [0, 1] : dx^ {ui,U2) ■= e{ui) + e(u2) — 
2inf„g[„j^„2,«iVM2] ^(^)' which becomes a true metric on the quotient space 
Te := M+I^g. Then by Lemma 3.1 in [14], for each e G [0, oo) with sup{t > 
0:e(i) > 0} < oo the metric space (Te,dT^) is a compact ordered M-tree. 

For e G [0, oo), let [0] := {u £ [0, oo) : e{u) = 0} and define the map 

(2.1) T:C^+[0,oo)^r°°*'i''^, 

which maps an excursion to a rooted linearly ordered M-tree by T(e) := 
(Te, ^Te, [0], <e°), with the linear order <g™ defined by ui <i'° U2 iff inf [ui] < 
inf[ti2], here [u] is the set of elements equivalent to u. The map T is contin- 
uous (Lemma 3.1, [14]). 

A rooted linearly ordered tree is finite if it has finitely many branch 
points and leaves. Let TPg""*''™ denote the subspace in T™°t'ii'i of finite rooted 
linearly ordered trees, and fix a speed a > 0. The linear order allows one to 
define a depth-first search on the set of its leaves and branch points. Depth- 
first search is a function that starting from the root visits every point of 
this tree while respecting its linear order by traversing the tree in such a 
way that its first visit to x comes before its first visit to y if x y. If one 
associates a speed a with this traversal and assigns a function which records 
the height of the tree at every time point of the traversal, then as a result 
one obtains a function e G C^+[0,oo). 

This defines a contour process map C('; c) : Tg°°*''"^ C^+[0, oo), which 
associates with each T G T™"*''™ an excursion C{T;a) G C^+[0,oo). It is 
easiest to see the correspondence of a finite rooted linearly ordered tree and 
its contour process from its planar representative (see Figure 2). Typically, 
one sets the speed a = 2, and for all a we have T o C = id,C o T = id. A 
formal description of the contour process can be found in Section 6.1 in [21]. 
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Genealogical point-processes. We next consider finite rooted linearly or- 
dered trees spanned by a population alive at a fixed time. Such trees are 
balanced, that is, have all the leaves at the same height. Given a family tree 
of a branching process and a fixed time t>0 this is the subtree of the fam- 
ily tree containing only information about the genealogy of the individuals 
alive at time t. That is, it is defined by the collection of mutual distances 
between the individuals alive at time t. This collection of distances forms an 
ultra-metric space, that is, for all xi, X2, a^s with d{p, xi) = d{p, X2) = d{p, X3) 
we have d(xi, X3) < d{xi,X2) V ^(2:2, X3). 

For a fixed t > we can record these distances in a point process. The 
linear order gives a linear order on the individuals alive at time t. We can 
represent these individuals as points along a line at height t with equal 
spacing between them. We can record the mutual distance between each 
neighboring pair of individuals as a point equally distant from the points in 
this pair at a height smaller than t. More formally, given t > and {T,p) G 

^root.lin ^ let 

(2.2) Qt{T,p):={xeT:d{x,p)<t} 
and 

dQt{T,p) ■.= {x€T:d{x,p)=t}. 

Define a map 'P*(*;?) which sends an element in qpg""*'^™ to a point process 
on [0,00) X [0,00) as follows. Given {T,p) G qpg""*''™, dQt{T,p) is an ordered 
finite set, so we can write dQt(T,p) := {xi, . . . ,x^qq^^x,p)} assuming that 
xi<i-X2<i"^---<i"^x#aQ,(T,p). Let 

V\{T, p); := {(k, t - id(xi, x,+i)); i = 1, . . . , #dQt{T, p)-l}. 

The set 'P^{{T, p);q) is a point-process representation of the tree dQt{T,p). 
Typically the spacing between the points is ? = 1 . 

Note that mutual distances of all neighboring pairs is sufficient to recon- 
struct mutual distances between all pairs. For any Xj,x; G dQt{T,p) such 



(112) (112) 




Fig. 2. Illustrates mapping of a finite linearly ordered tree to an excursion. 
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that Xi < Xj the distance d{xi,xi) = infj<j<yfc<i{(i(a;j, x^)}. More detailed 
description of the point-process representation can be found in [22] . 

Letting the time t > vary, this cohection of subtrees of individuals alive 
at time t defines the full family tree. That is the set T = IJt>o dQt{T, p) with 
the linear order and distance metric of (T, d, p, <^™) preserved in the subtrees 
spanned by dQt{T, p). Hence, the collection of point process {P*((T, p);(;)}t>o 
gives a complete representation of the metric space (T, d, p, <'"^). 

Gromov-HausdorjJ distance. We finally recall the appropriate measure 
of distance between two M-trees. The Hausdorff distance between two com- 
pact subsets Ki and K2 of a complete metric space {E,r) is defined as 
duiKi,K2) := inf{e >0:KfD K2,K^ 5 Ki}, where for e > and K <Z E 
closed, := {x G E:r{x,K) < e}. The rooted Gromov-HausdorJJ distance 
between two rooted M-trees (Ti , px^ ) and (T2 , PT2 ) is then defined as 



(2.3) dGH-°t((7'i,PTi),(r2,pT2)) :=infdH ' (T^', Ts) V dz(pTi , PTj). 



where the infimum is taken over all rooted R-trees (T{,p^^) and {Tl^^p'j,^) 
that are root-invariant isomorphic to (Ti,ptJ and {T2,pt2)^ respectively, 
and that are (as unrooted trees) subspaces of a common metric space (Z, dz)- 
By Theorem 2 in [13], (T™"* , dGH^o* ) is a Polish space. 

Notational conventions. Throughout the rest of the paper the symbol 
=^ will always denote convergence in distribution in the underlying process 



spaces. These are: 

• for population size/total mass processes: cadlag paths with Skorohod topol- 



• for forest-valued processes: (T™°* , dQjjroot ) of (isometry classes of) compact 
rooted M-trees with rooted Gromov-Hausdorff topology [compare (2.3)]; 

• for contour processes: nonnegative continuous functions with the topology 
of uniform convergence on compact sets; 

• for point process: locally finite measures on [0, 00) x [0, 00) with the weak 
topology (i.e., the topology given by convergence of all bounded continu- 
ous functions). 

3. Catalytic branching model. In this section we introduce the model 
and the family forests with their contour processes and point-process rep- 
resentation that will be used to obtain our results. We begin in Section 3.1 
by defining the catalytic branching particle model and the family forests 
of its two populations. We define the contour process and the collection of 
point-processes associated with these finite family forests. We then define in 
Section 3.2 the suitably rescaled versions of these processes, and the diffu- 
sion limits of interest. Finally, in Section 3.3 we recall some known results 
for these processes which are mainly for the catalytic population. 



n- 




■00 
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3.1. Definition of the model and its family forests. The catalytic branch- 
ing model consists of two different populations of distinct individuals: the 
catalyst population and the reactant population. The pair of populations 
evolves as a Markov process {'i]t,(,t)t>o according to the following rules. 
The catalyst population 77 is a classical continuous-time critical branching 
process: every individual has an exponential lifetime with parameter bi af- 
ter which it is replaced by or 2 offspring with equal probability. The 
reactant population ^ evolves analogously except that the exponential life- 
time distribution of each individual is replaced by a lifetime distribution 
F{t) = 1 — exp{— J^^^ b{u) du) where s is the birth time of this individual 
and b{u) equals 62 times the total number of catalyst individuals at time u. 

The total number of individuals in the catalyst, reactant population is 
a continuous-time Markov process with values in (jnN)^ if we associate mass 
m with an individual. Typically one sets the mass of a single particle m = 1 . 
However, to obtain a reasonable limit of the rescaled catalytic branching 
model we will need to scale down a single particle's mass contribution. 



Definition 3.1 (Total mass processes). 

The catalyst total mass process ry*°* = {r]l°^)t>o is a critical binary Galton- 
Watson process with constant branching rate 61 



rjr ^ ivr^m) ^ I "^C^""' each at rate h,^, 
[ 7]^ —m, 2 m 

• given r/*°*, the reactant total mass process = ('Ct°*)t>o is a critical 
binary Galton- Watson process with time-inhomogeneous branching rate 

- ; -) ^ :L ^ each at rate ^62,- 1-. 
L ?f ~ ''''1 ^ 

We introduce the random family forests of the catalytic branching particle 
populations encoded as elements in T"^""* as follows: 

Definition 3.2 (Forest- valued catalytic branching particle models). Let 

^f°'--=( U {^}^d„pA, 



'-for 
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Remark 3.1. Recall that criticality of the catalyst process implies that 
r/*°* will almost surely get absorbed at 0. Let 

r°'i :=inf{t>0:??<*°*=0}. 

Notice that after the catalyst mass process is absorbed at 0, the reactant 
mass process gets absorbed as well. That is, in the reactant family tree the 
most recent common ancestor of any two points at height greater than or 
equal to T^'^ has height smaller than T^'^, or equivalently, after T"'^ the 
branches simply extend to oo. In the following we impose the convention to 
cut off all the branches of at height T^'^, that is, is the subspace 
of the reactant family forest consisting of all points in the reactant family 
forest whose distance to the root is at most T^'^. 

The forests 77'^°'^ and (with the convention given in Remark 3.1) have 
finitely many leaves and branch points, almost surely. We define their con- 
tour process representation: 

Definition 3.3 (Contour processes of the finite family forests). Let 
B := {Bu)u>o = C{v'°'; <J), C := (C„)„>o = C{^'°'; a). 

For the collection of genealogical point processes of the family forests in 
the catalytic branching model we define: 

Definition 3.4 (Genealogical point processes). Let 

n = {n*}t>o, n*^p*(7?f-;?), H = {H*}i>o, ^v\t';<i)- 

3.2. Resettling of the model. We now consider a family (f/",^") of cat- 
alytic branching particle models indexed by n, where the parameter n means 
that the initial number of individuals is increased by a factor n and the 
branching rate of the catalyst is sped up by a factor n. 

We will define for the rescaled model the total mass process, the family 
forest, the contour process and the collection of point processes. We begin 
with the rescaled total mass processes. 

Definition 3.5 (Rescaled mass processes). The rescaled total mass pro- 
cess (fytot:"^ ^tot.") is a continuous-time Markov chain with (7?o°*'"', Co°*'") = 
(1, 1) and 

tot,n ^tot,n\ 



^~tot,n ^tot,n^ 




~tot,n^ |tot,n ^ 1\ ^ ^^^g 1^2^^ ~tot,n|tot,n^ 

nj 2 
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The rescaled population processes also have associated family forests for 
which we use the following notation: 

Definition 3.6 (Rescaled forest- valued processes). Let 7^*^™'" be the 
family forest of rooted M-trees associated with fp , and let .ff™'" be the 
family forest of rooted M-trees associated with 

Recall that we imposed the convention that the branches of are cut 
off above height 

(3.1) r°'":=inf{i>0:?7f*'" = 0}. 

Since the branching rate of the rescaled population process is accelerated 
by a factor of n, the edge lengths in the forest are shortened typically by a 
factor of ^. 

Remark 3.2. Since we can always rescale time by a constant without 
any effect on the shape of a forest we shall henceforth assume that 61 = 1. 
For many purposes, in particular if the catalyst medium is given, we also 
may assume that 62 = 1- 

With probability of order O(^) the height of a Galton- Watson tree is of 
order 0{n), and given a Galton- Watson process is still alive at a time of 
order 0{n) its population size at that time is of order 0{n). Hence, starting 
initially with n particles and speeding up time by a factor of n yields in the 
limit a Poisson number of trees, each having total population size of order 
0{'n?). Furthermore, since branching is sped up by a factor of n all edges are 
of order C'(^). As a consequence, in order to find a nontrivial limit contour, 
in the nth approximation step we need to traverse the rescaled forests fy*^™'"- 
and at speed a = 2n. 

Definition 3.7 (Rescaled contour processes). Let 

= {B:)u>, ^ C(?f'"; 2n), (7" = (C':)„>o = C(e^-'"; 2n). 

Furthermore, to keep the genealogical point processes representing the 
rescaled forests jf™'"^ and in a compact set, we need to let the spacing 

Definition 3.8 (Rescaled genealogical point processes). Let 

n" 



{n*'"}t>o, 

1" jt>0) 



n 



t,n 



n, 



bt.n . 



1 



n 
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3.3. Limit of the mass processes and of the catalyst family forest. Some 
of the rescaUng hmits for the introduced objects, in particular all those which 
are associated with the catalyst process, are well known. We next recall these 
results. 

With standard techniques one can show that there exists a Markov process 
{X,Y) with paths in Pir+xR+IOjCxd) such that 



(3.2) (77*°*'",e*°*'") ^ (x,n 

n- 



(see, e.g., [16]). Moreover, {X,Y) is the unique strong solution to the fol- 
lowing system of stochastic differential equations 



(3.3) 



dXt = VYtdw,^, 



dYt = VX^tdW^ 



Y 

t 



with initial value {Xo,Yo) := (1, 1), and where = {W^)t>o and = 
{Wf^)t>o are two independent standard Brownian motions on the real line. 
We refer to {X, Y) as the catalytic Feller diffusion. 

Another useful fact about the catalytic Feller diffusion process is given 
by an explicit expression for the probability that {X,Y) will hit the y-axis 
before the x-axis. Let for 6 >0 

■.= mi{t>0:Xt = 6}, 

(3.4) 

p° ■.= mi{t>0:Yt = 0}. 
Then, as calculated in [20], 

^<^°<^°'^(^|-)"' 

Next we recall what is known about the genealogical forest of the catalyst. 
It was shown in [1, 2, 3] that a suitably rescaled family of Galton-Watson 
trees, conditioned to have total population size n, converges as n — > oo to 
the Brownian continuum random tree {CRT). With similar arguments one 
can get a result for the suitably rescaled catalyst forests; in fact, our proof 
here contains this as a special case. Also, all the limits of the contour process 
and genealogical point processes have been identified earlier, that we briefly 
recall. 

Let \(3\ = {\l3\u)u>o be a reflected Brownian motion and I = (^m)«>o its 
local time at level 0. Let (-~^{t) = inf{u > 0:^^ = t} denote the inverse of 
the local time of at level 0. We have 

(3.6) 5"=^2|/3|.,,-i(,). 

It follows from (3.6) that there exists X^°^ G T"^""* with 

(3.7) rf"''"^ =^ X^""'. 

n^oo 
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Moreover, X^"^ equals in distribution T(2|/?|.^^-i(x)) [recall that the map T : 
C^+[0,oo) — > TT™"*'^™ sends an excursion to a rooted linearly ordered com- 
pact M-tree]. 

Note that if Xq = 1 then consists of a Poisson(j) number of family 
trees. For the associated genealogical point process this translates into a 
Poisson(i) — 1 number of points at depth at least t separating these trees 
in the forest. For a single tree the following is known by Theorem 5 in [22], 
{tn} is such that t„ — > t for some t G M+ then 



+ 00 



n— »oo 

The distribution of vr'^'* is the mixture over the law of Xt of a Poisson point 
process whose intensity measure, for a given value of Xt, is 

(3.8) >^P^\M X dh) = l[o,x,] di<S>l^o,t)j^^- 



4. Results. In this section we state limiting results for the pair of family 
forests of rescaled catalytic branching populations. Since evolution of the re- 
actant depends only on the evolution of the catalyst total mass process, we 
can analyze the conditional distribution of the reactant family forest given 
the values of the catalyst mass process. This is analogous to a "quenched" 
analysis for a process in random environment — the catalyst total mass pro- 
cess playing the role of the random environment. 

We start Section 4.1 with the claim that the family of rescaled reactant 
forests given the catalyst total mass process has a limit forest. To determine 
its distribution, and thereby characterize a unique reactant limit forest, we 
first consider the environment up to the time in which the catalyst stays 
above an arbitrary positive threshold. We derive the limit of the rescaled and 
"truncated" contour processes, and we describe the reactant limit contour 
from this family of diffusions. Based on the latter we derive the rescaled 
genealogical point processes of the reactant. Section 4.2 gives the result on 
joint convergence of the catalyst and reactant family forests to a limiting 
pair of forests. Finally, in Section 4.3 we discuss the core differences between 
the genealogical structures of these two limiting forests. 



4.1. Reactant in a fixed catalytic background. By (3.2), we can realize 
the rescaled catalyst total mass processes and the limiting Feller diffusion 
X = {Xt)t>o on a common probability space such that for all T > 



(4.1) 



GENEALOGY OF CATALYTIG BRANGHING MODELS 



13 



Hence, branching rates of the reactant processes will be given by a 

sequence of functions in Skorohod space which converge uniformly on com- 
pacta to a continuous hmit function X . We study the behavior of the rescaled 
reactant forests under assumption (4.1). 

The first result states that given the catalyst medium, the family of 
rescaled reactant forests has limit points. We use (; •) to denote the condi- 
tional law for the reactant with fixed realization of the catalyst environment. 

Proposition 4.1 (Existence of reactant hmit forests). Let fj^"^'^ and X 
satisfy (4-1)- Then the family {(^^°'''"; r/*°*'"); n G N} is relatively compact. 

To characterize the limit {Y^""; X) of forests {(1^°'''"; ?7*°*'"), n G N} we use 
the contour process and genealogical point process representations. 

4.1.1. The reactant limit contour in a fixed catalyst medium. In order to 
obtain convergence of the rescaled contour processes to a diffusion process 
we restrict our consideration only up to the first time the catalyst mass falls 
below an arbitrarily small threshold (5 > 0. Let 

(4.2) r^''":=inf{t >0:r?f*'"<(^}. 

Recall that Qt defined in (2.2) takes a forest and cuts off the portion that 
lies above height t. Let denote the reactant trees of the cut off 

at f^'"" (rather than at 7"^'°) 

t, .— LJfs^n^!; ) 

and C"^'" its contour process 

C'^." = (C^-«)„>o:=C(f-'^'";2n). 

The first result describes the behavior of the limit of the rescaled reactant 
forests in a catalytic environment that is stopped at T^'". 

Theorem 1 (Limit of the reactant contour process). Assume (4-1)) md 
fix 5 > 0. Consider the operator {A^ ,V{A^)) with 

(4.3) A'f{c):=2^^j'J{c) 
with reflection on the boundary, that is with domain 

(4.4) V{A') := |/ G CH[0,r']) : G C\[0,t']), f\,^,s} ^ o}. 
Then the following holds: 
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(i) The {A^ ,'D{A^)) -martingale problem is well posed. 

(ii) If is the solution of the {A^ ,'D{A^)) -martingale problem then 

(4.5) (C"^'";??*°*'") ^(C^X). 



n— »oo 



Remark 4.1. By standard arguments (see, e.g., [25]) an operator in 
divergence form such as A^ specifies a unique diffusion C,^ with a weU defined 
local-time process i\C^) := {ii{C^))u>o at all levels t > 0. 

Recall that T:C^+[0, oo) qp™"*'^™ maps an excursion to a rooted lin- 
early ordered compact M-tree. Together with Theorem 1 we can immediately 
conclude the following. 

Corollary 4.2. For all 6>0, and limit points Y^"' of {(l^™'"; f)*"*'"); 
neN} 

(4.6) {Q,s{Y'°');X)It{C';X). 

This allows us to uniquely identify the reactant limit forest. 

Theorem 2 (The reactant limit forest exists). There exists Y^°'^ £ T™°* 
such that 



^|for,n. ~tot,n) ^^(yfor. j^y 



This now raises the question whether we could let (5 — > and obtain a 
limiting diffusion. Recall from (3.4) that is the first time the catalyst 
becomes extinct, and is the first time the reactant becomes extinct. As 
stated in (3.5), the probability of the reactant going extinct before the cat- 
alyst is positive. In case the reactant dies before the catalyst {p^ < t^}, the 
reactant is always in a strictly positive catalytic background. Hence, on this 
event, the limiting contour process is well defined by Xt E {0,oo), for all 
t £ [0,^*^], as a diffusion with generator 



Af{c):=2{^J'] (c), fGViA), 



on the domain 



V{A) := |/ G CH[0, rO]) : ^ f G Cf.^^o] [0, oo), /'(O) = 

However, on the event that the catalyst dies before the reactant {p^ > r^}, 
it is clear that there does not exist a well-defined limiting operator for A^ 
as 6^0. As will be shown in Corollary 4.7, on this event it is not possible 
to define a diffusion process such that the tree below its path has the 
distribution of the limiting forest Y^"^. 
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4.1.2. The limit reactant point process in a fixed catalytic medium. The 
point process representation of the genealogy is useful in those cases when 
we can give a simple description of its distribution. For both the catalyst 
and reactant genealogy one can show that given the total mass it can be 
represented by a Poisson point process with a specified intensity measure. 
Our next result describes the reactant limit genealogical point process. 

Theorem 3 (The reactant limit genealogical point process). Assume 
(4-1)- Por any t > 0, and any sequence itn > 0)„gN such that tn — > t, 

n^oo 

n — ^oo 

where for a given path X = {Xt)t>{), the point process vr* consists of: 

- rate {f^ Xgds)"^ Poisson point process at height whose points repre- 
sent a separation of distinct trees in the forest, 

~ Poisson point process on M"*" x M"*" whose intensity measure conditioned 
on Yt is 

(4.7) ^^'' W X dh] = l[o,y,] (£) di l(oMrOApO) [h) dh. 

\Jh -^sds) 

We have only constructed a map 'P*(';<j) which sends an excursion of 
finitely many local maxima to a genealogical point process. To extend this 
map to excursions of infinitely many local maxima one would either proceed 
via a limit procedure or look at the point process of maximal depths of 
downward excursions. We next relate the reactant limit genealogical point 
process to the point process of depths of downward excursions corresponding 
to the limit contour. 

Proposition 4.3 (Relation between the limit point process and C*^). 
Fix 6 > 0, and let be the solution of the well-posed martingale problem 
{A^,'D{A^)). Given X := {Xs)s>o, ^ > and t G iO,T^), the point process 
IT* equals in distribution the point process of maximal depths of downward 
excursions of below level t indexed by the local-time process {iu{C^))u>o 
ofC'. 

4.2. Joint law of the catalyst and reactant. The result of Theorem 2 
can be extended to joint convergence of the catalyst and reactant rescaled 
forests, where for pairs of forests we use the product of Gromov-Hausdorff 
topologies. 

Theorem 4 [Limit of (catalyst, reactant)-forest]. We have 

(4.8) (fj^°'''', 1^°'''") =^ (X^", y^™). 

n — ^■oo 
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The result in Theorem 4 becomes relevant, for example, if we want to 
sample groups of individuals from both populations and to investigate the 
pair of matrices of mutual genealogical distances of sampled individuals. 

4.3. Differences between the reactant and "classical" forest. Let |/?| = 
(|/5|u)'u>o be a reflected standard Brownian motion, and for all t > 0, := 
{^u)u>o its local time at level t. As before let i~^{s) := mi{u >0:£^ = s}.lii 
the following we refer to 



as the Brownian forest. Recall from (3.6) together with (3.7) that Z equals 
in distribution the catalyst forest X^°'^. 

We now discuss the qualitative difference between the limiting forest Y^°^ 
(with stochastically evolving branching rates), and the Brownian forest 
(i.e., with fixed branching rates). We shall focus on the following aspects: 

• differences in the tree structure due to inhomogeneity of the random en- 
vironment (evolving branching rates); 

• behavior of the forest Y^""^ at time of extinction of the random envi- 
ronment (given the reactant survives the catalyst). 

These two items can be considered from quenched and annealed points of 
view. Of course, for a statistician observing populations the annealed point 
of view is more relevant. 

4.3.1. Differences due to the inhomogeneous environment. We first de- 
scribe how in the quenched picture the reactant forest Y^°^ can be obtained 
from the standard Brownian forest Z^°'^ by height dependent "stretching" 
of the tree metric expressed in terms of the total mass process of the fixed 
catalyst medium X. The next result compares the distribution of genealog- 
ical distances using the point-process representation. Recall from (3.4) that 
r° is the first time that Z^°'^ becomes extinct. The proposition compares 
in law the distances between points at a fixed level t in a reactant forest 
dQt{Y^°^), with the distances at another level st{t) in a standard Brownian 



Proposition 4.4 (Stretching the tree metric). Assume we are given 
a Brownian forest (Z^°'^ ,dzfoi). For any G (0,00) and continuous func- 
tion x:[0,T^) M+, for a fixed t G (0,r'^) define an increasing function 



Z'-:=T(2|/3|.^,-.(i)) 



forest 9Q,,(i)(Zf-)- 



s^:[0,t]^[0j^xsds] by 
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and let {Sf)~^ : [0,/q Xgds] [0,t] be its inverse. Then the following holds: 

((9Qi(y^-),(iyfoO;^ = rE) ^ (aQ,.(i)(Z^"),2 • (4)-i(id^foO). 

Theorem 3 allows us to compare certain statistics of the reactant limit 
forest with those of a "classical" Brownian forest. The latter appears as the 
scaling limit of the uniform discrete forest with a fixed number of vertices 
as the number of vertices tends to infinity and may therefore be seen as the 
uniformly distributed compact rooted forest. 

The following result states that provided we compare the reactant limit 
forest with the "classical" Brownian forest which has the same expected 
number of trees, the reactant limit forest has a smaller probability that two 
"randomly" picked individuals at a fixed time belong to different family trees 
than in the "classical" forest. 

What does it mean to "pick individuals at random?" Recall that the 
map T takes an excursion and sends it to a rooted compact M-tree. Fix 
(T,p) G TP™"*, and let h{T,p) denote the height of T in terms of the largest 
distance of points in T from the root. Assume that there exists an excursion 
C G C^+[0, oo) with T(C) = {T,p) such that the local-time processes ^*(C) = 
{£\Os)s>o exist for all t G {0,h{T,p)), and denote the inverse by (^*(C))"\ 
that is, {i\C)yHx) := mf{u > 0:£^(C) > x}. Let L*(C) := lim,_oo4(C) < 
oo. Define : [0, l]^T and /i* by 

H\u) := i£\C))-Hu ■ L*(C)), /i* := iH%X, 

where A is Lebesgue measure on [0,1] and HlX denotes the push forward 
of the Lebesgue measure under H^. Obviously, ^* can be thought of as the 
"uniform" distribution on the set dQt{T,p). 



Proposition 4.5 (Comparison of the reactant with the classical tree). 
Let (X*^™, y*^™) be the catalytic branching forests with branching rates hi = 
62 = 1, and let Z^°^ be the forest associated with a standard reflected Brown- 
ian motion 2\(3\ stopped at £~~^{z) = inf{n > : £u(2|/3|) = z} for some z>0. 
Fix t G {0,h{Y^°') A h{Z^°')), and let Yt = Zt and p^'^ and be the ''uni- 
form" distributions on dQt{Y^°'^) and dQt{Z^°^) , respectively. Assume fur- 
ther that the expected number of trees alive at height t are the same in Y^°^ 
andZ^°', i.e. that zit)-'^ =E[{J^ Xsds)-'^]. Then 



E 



(9Qt(yfoO)2 
<E 



(/i*'^)®2(dn, du')l{dYfo. {u, u') = 2t} 

{p^'^)®^{du,du')l{dzior{u,u') = 2t} 
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4.3.2. On top of the reactant limit tree. In the region where the catalytic 
environment is about to go extinct the reactant has very few branching 
events. Hence the structure of the forest cannot be compared to the Brow- 
nian forest by a finite map of distance stretching. As a consequence the 
reactant hmit forest will have infinite F-length which is in contrast to the 
Brownian forest which is known to have finite P-length. 

To be more precise, let for a given (T, d) £T and a finite subset T' C T, 

\\{T,d),T'):= id{a,b)f. 

a,beT' : ]a,fe[nT'=0 

We then say that {T,d) is of finite (infinite) -length if there exists a finite 
(infinite) random variable \^{iT,d)) such that for every sequence (T^)„gH of 
finite subsets of T with inf{e >^:(T'Y = T} — > 0, 

(4.9) \\{T,d),Tl,) ^j\{T,d)) in probability, 

where A" denotes the e-neighborhood of A C T. 

Lemma 4.6 (Infinite versus finite F-length). Let (X^™,yf°^) he the cat- 
alytic forests with branching rates &i = 62 = 1; o^nd Z^°^ a standard Brownian 
forest. Then 

P{l2(yf°r) = 00} = P{r° < /} > 0, 

while 

l2(Z^°'^) < 00, a.s. 

From Lemma 4.6 we immediately conclude that there cannot be a diffu- 
sion (^^ such that the forest below the path of a realization of (^^ equals in 
distribution the reactant limit forest. 

Corollary 4.7. Let Y^°^ be the reactant limit forest, and C be a random 
element in C^+[0,oo) with sup{t > 0:C{t) / 0} < 00, a.s., such that T{C,) 
equals in distribution Y^™ . Then given that {t'^ < p*^} the total quadratic 
variation of C, is infinite. In particular, on the event {tq < po}, Y^™ cannot 
be associated with the path of a diffusion process. 

4.4. Extensions to more general processes. In this paper we are mainly 
interested in the catalytic branching model as one of the three principal 
types of neutral branching models, as pointed out in the Introduction. If 
one assumed a perspective of branching processes in a random medium one 
could place the emphasis differently and start with more general Markovian 
dynamics for the catalyst process. One would assume that the scaling limit 
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of the total mass of the catalyst and its genealogical tree exist and that the 
limit processes of the total mass of both the catalyst and reactant have paths 
in C[o,oo)([0; oo)) with compact and connected support. Under these assump- 
tions the techniques we developed in our proofs would imply convergence of 
the quenched total mass process of the reactant to a limit diffusion process, 
convergence of its quenched genealogical forest to a limit forest, as well as 
the convergence of the annealed bivariate objects. Similarly, Proposition 4.4 
on stretching of the tree metric also remains valid. 

Interesting examples could be provided by catalysts which are total mass 
dependent branching processes, that is time inhomogeneous models with 
branching rate h{'tf°^) where /i[0, oo) — > [0, oo) is a locally Lipschitz function 
with /i(0) = 0. 

If we are willing to give up results on the bivariate dynamics, we could 
also allow for the following. Suppose we are given the dynamics of a process 
of masses converging to a continuous path limit and still have all quenched 
results for the reactant forest. That is, assume that we have some scaling 
function r : N ^ such for all T > 

I tot.n I ri 



t<T 



lr{n)t - ■ n-*x> 



holds for some X E C[o^oo)([0) oo)) with X~^{(0,oo)} almost surely a finite 
interval. Then Theorems 1, 2 and 3 hold again as well as Proposition 4.4. 
More generally the branching rate of the reactant could be 5(^*°^'") for 
continuous functions / vanishing at most at zero. 

5. Proof of tightness (Proposition 4.1). Throughout this section we as- 
sume that realizations of r)*°*'", n G N, and of X on a single probability space 
are fixed such that (4.1) holds. 

To prepare the proof of Proposition 4.1 we specify a compact set of rooted 
compact M-trees which we will use for proving tightness. It will be helpful 
here to use genealogical terminology. For each (T, p) G T''°°* denote by dT := 
T \ {j^^rp[p,x[ the set of leaves of T, and by h{T,p) := sup^^zT d{p,x) the 
height of the tree. We introduce the set A!i_^{T,p) of ancestors of the time 
t population a time e back as follows: for all e > let 

Ss{T,p) := {p} U {x G T:3y G T, with x G [p,y], and d{x,y) > e} 

be an e-trimming of the tree (T, p) . In particular, if h{T, p) <£ then Sg^iT, p) = 
{p}. Then for given {T,p) G T™°* let 

A\_, (T, p) := Se {Qt (T, p) ) n dQt-e {T, p) 

be the set of ancestors at time i — e of those individuals that are alive at 
time t. It follows from Lemma 2.6 in [13] that Se{T,p) G T^°°' and hence 
#A\_,{T,p)<^. 
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The following result says that the family of rooted compact M-trees whose 
number of ancestors a positive time e back are uniformly bounded is pre- 
compact. 

Lemma 5.1 (A compact set of rooted M-trees). Fix a family {L^;m £ 
N} of positive integers. Then the set 



(5.1) 



(r,p)Gr°°*; 



l<fc<[2('"+l)/i(T,p)J 

is compact in T^°°*. 

Proof. Recall from Lemma 2.5 in [13] that a family F is pre-compact 
in (T™°* , c^QHroot ) if for every e >0 there exists a positive integer n(e) < oo 
such that each T gT has an e-net with at most n(e) points. 

Fix e > and let (T, p) G F. Then with tuq := mo(e) = [— log2(e)J , 

^(^) - U 

fceN 

is an e-net. Indeed, if x G T, then for Bi;{x) := {y £T: d{x, y) < e} 

R / /,2-('"0+i)[2('"0+l)/i(x)J X 
X G -«£(A2_(^|j+i)|^2(mo+l)/i(a.)_ij J- 

Moreover, since T G F, j^N{e) < L™" = n{e). Since F is closed, compactness 
follows. □ 

Proof of Proposition 4.1. We need to show that for all 7 > 0, we 
can find a compact set F : = F^ C T™"' such that for all n G N, 

(5.2) p({|for,n ^ r}|r/*°*'") > 1 - 7, 

provided that f/*°*''" and X satisfy (4.1). 

As candidates for such a compact set we consider sets of the form (5.1). To 
verify (5.2) calculate for fixed nondecreasing positive integers {L"*;m G N}, 

P| #^2=£+i!(fc-i)(e'°'''") < ^'"-Vm G N|r?t°t'"| 



■l<fc<[2('"+i)f'0."J 



"^>l l<fe<[2(™+i)f'0."J 

l<fc<[2(™+l)f 0'"J 



~tot,n 



m>l 



-tot.n 
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By Theorem 5.1 in [7], (t^tIj^^; r7*°*'")£>o is the number of blocks in a 
time-inhomogeneous Kingman coalescent at rate l/ff^^ . Moreover, \i K := 
{Ks)s>Q is the number of blocks in a Kingman coalescent then limgj^o ^-^e = 2 
almost surely as well as \\m.^^Q£K{Ki,) = 2, see page 27 of [4]. Hence there 
exists a constant c > such that 

l<A:<[2('"+i)f'0."J 

>i--i:- y. 

— Tm 



L — < J^m rA;2-('"+l) ~tot,n , 

(^5_3~) rn>l l<fc<[2("i+i)rO>"J J(fc-l)2-(™+i) ^« "'^ 

> 1 - c y — 2('"+2) f min 77*°*'") I 2(^+1)^0'" I 
^ 1 - c V J-2(™+2) f min X,) I 2('"+^)r0j . 

n^oo ^ L"^ ^sef0,T0-2-(™+2)l ^ 

m>l ^ ' ' 

Here we have used that for all m G N and fc G {1, . . . , L2(''"+^)r°'"J }, 



(fc-l)2-('"+l) J(fc-l)2-(™+l) 

> 2-('"+2) . ^tot,n^ 

sG[0,fo."-2-('"+2)] 

Since r'' < cxo and min^^jQ ^o_2-{m+2)] > 0, for all m G N and all real- 
izations of X, we can choose the family of positive integers {L'^,m G N} 
suitably large, for example, L™- > (4/(7)'"(min^g[Q ^o_2-{m+2)] X^)"^ with < 
g = 7/(7+ 1) < 1/2, such that the right-hand side of (5.3) is greater than or 
equal to 1 — 7, and (5.2) follows. □ 

6. Proof of the limit for the reactant contour (Theorem 1). In this sec- 
tion we give the proof of Theorem 1. We first show that given a realization 
of the catalyst total mass process the reactant contour process is associated 
with a Markov process, and we derive its generator. We also give a repre- 
sentation of this Markov process as a random evolution process. By this we 
mean that it moves at constant velocity for a random time, then changes 
the sign of its velocity and proceeds at constant velocity for a random time 
again. We use this representation to prove that the suitably rescaled contour 
processes of the truncated reactant forest converge toward a limit contour 
process which is characterized as the solution of a well-posed martingale 
problem. 
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Throughout this section a reahzation rf°^ := (?7s°')s>o G -Dn[0,c>o) of the 
catalyst path is fixed. Recah the reactant contour process C := {Cu)u>o from 
Definition 3.3, and let its slope process V := (Kt)it>o be defined by 

Vu:=\ slope(Ct.) G E'siope 

with 

-E'slope := {— l^+lji -E'cont := [0,T°'^], 

where the slope of the root, branch points and leaves are defined in such a 
way that (K)n>o has cadlag paths and T"'^ := inf{t > : rf"^ = 0}. 

The next result states that the functional pairing of the height of the 
contour with its slope is a Markov process. 

Lemma 6 . 1 (Markov property of the contour process) . The process (C, V) 
{Cu, Vu)u>o on -Econt X -E'siopo is a Markov process whose generator is the clo- 
sure of the operator 

(6.1) Aficv) = 2v-^Jic,v) + 7jl"'[fic,-v) - f{c,v)] 

for all f G ^{A), where 



(6.2) V{A) = |/ G C7i'0(S,o,t X E,iopo) 



dl 
dc 



9(-Econt)xi?siope 



Proof. Recah that C(-; a) : Tg°°*'''° ^ Cr+[0, oo) maps a rooted M-tree 
to an excursion from Figure 2. We first show that the lengths of the line 
segments of the contour process are distributed as a sequence of independent 
variables stopped at the first time their alternating sum falls below 0, and 
then we use this to obtain the Markov property and to identify the generator. 

Step 1 (Sequence of lengths of the line segments). Recall from Figure 
2 the assignment of a contour process to the representative of a Galton- 
Watson family forest embedded in the plane. Each piece of the contour 
process with constant slope sign corresponds to a sum of a number of life- 
times, so the distributional relationship between the different line segments 
is not obvious. We shall make use of the fact that the reactant process 
can be represented in two ways without changing the distribution of its to- 
tal mass process or the genealogical distances between individuals, either 
as continuous-time binary Galton-Watson process, or as a birth- and- death 
process: 

• In the continuous-time Galton-Watson process the branch points occur 
at rate 7?j;°*, and the number of offspring at each branch point is or 2 
with equal probability. 
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• In the birth- and-death process each individual dies at rate 3^*°*! 
during its hfetime gives birth to new offspring at rate ^77*°*. 

If in the Galton- Watson process at each birth time we choose to identify 
the hfe of one of the offspring as a continuation of the hfe of its parent, we 
obtain the birth-and-death process. 

The family forest of the Galton-Watson process has a canonical planar 
embedding coming from a linear order on its vertices induced by the linear 
order of the tree as in Figure 3(a), while the family forest of a birth-and-death 
process with that same linear order on the vertices has two canonical planar 
embeddings. In Figure 3(b) we always choose to identify the continuation 
of the life of the parent with the life of the offspring of higher linear order, 
and the branch of the offspring is always drawn to the left of the branch of 
the parent. In Figure 3(c) we always choose to identify the continuation of 
the life of the parent with the life of the offspring of lower linear order, and 
the branch of the offspring is always drawn to the right of the branch of the 
parent. 

The key observation now is that since all three planar embeddings respect 
the same linear order on the vertices they also have the same contour process 
(compare Figure 2). 

In the birth-and-death process the line segments of constant slope corre- 
spond to a lifetime of exactly one individual. With the parent identification 
as in Figure 3(b) each line segments of negative slope corresponds to a life- 
time of an individual, while in the parent identification as in Figure 3(c) 
each line segment of positive slope corresponds to a lifetime of an individ- 
ual. Since lifetimes of individuals are independent this implies the constant 
slope line segments in the contour process are distributed as a sequence of 
independent variables stopped when their alternating sum first falls below 
0. 
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Fig. 3. Illustrates three different planar embeddings of the reactant family forest with the 
same linear order and a common contour process. 
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Step 2 (Identification of tlie generator). Given the branch rates (^/t°*)f>0) 
the law of the length / of a lifetime of an individual is 

where t* is the time of birth of that individual. Hence the law of the length 
of each segment is the law of the first point of a Poisson process with rate 
{ril°^)t>t* or {r]l°^)t<f if the slope of the line segment is +2 and —2, re- 
spectively. The alternating sign of Vu changes at rate r/^* where Cu is the 
current value of the contour process. In between the jumps of V^, Cu moves 
with speed of 2 units in the direction determined by Vu- Hence, the contour 
process paired with its slope is Markovian, and furthermore its generator 
agrees on T>{A) with the operator given in (6.1) and (6.2). Since T> is dense 
in C(-Econt X -E-siope) the generator is the closure of the operator (A,!)). □ 

We next notice that (C, V) is a random evolution, that is, a Markov pro- 
cess moving at constant speed in a direction whose sign changes stochas- 
tically. Specifically, for the pair (C, V) the change in speed is a counting 
process whose rate is governed by the catalyst mass process r]^°^. 

Lemma 6.2 (Random evolution representation). Let N := {N[u))u>o be 
a unit rate Poisson process, and consider the following system 

(6.3) Cu= r2V.dv, Vu = {-lf^^o'c:'^^ 

Jo 

in int(£'cont) x -E'slope o,nd with reflection on 5(i?cont) x -E-siope- There exists 
a unique random evolution satisfying this system, and its distribution is the 
same as the distribution of the contour process and its slope (C, V) . 

Proof. See Chapter 12 of [12] for the definition of a random evolution. 
Existence and uniqueness of a random evolution satisfying the system (6.3) 
follow from standard theorems on existence and solutions of a system of 
stochastic differential equations with continuous local martingales as differ- 
entials (see Theorem 3.15 in [23]). Equality in distribution of this random 
evolution with the contour process and its slope follows by simply compar- 
ing the generator of this system to the generator obtained in the previous 
lemma. □ 

We will also need the following easy consequence of the above arguments. 
Fix 6 > 0, and recall that T^'^ from (4.2) is the first time that the catalyst 
process started from 1 individual drops below 6. Also recall that Qrps,i from 
(2.2) is the map that takes a tree and cuts the portion of its branches that 
lie above height T^'^. 
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Corollary 6.3 (Truncated process). If the contour C(^*^™;1) solves 

(6.3) then C{Qrps,i{^^°^);l) solves (6.3) hut with the state space -Bcont re- 
placed hy El^, = [Q,T^^\ 

We next show that the rescaled random evolutions converge to the solu- 
tion of the well-posed martingale problem stated in Theorem 1. We rely on 
averaging techniques established for random evolutions. Throughout we fix 
realizations of 7]*°*'"^ ?i G N and X on a single probability space such that 
(4.1) holds and we choose a truncation parameter 5 > 0. 

Step (The rescaled random evolution system). Recall the rescaled 
reactant contour process C-^'" from Definition 3.7. Let ^,Xt — [0,^'''"], and 
define its rescaled |-slope process V^'"^ := | sign(slope((7. '")). 

Then Lemma 6.1 applied to the rescaled reactant populations implies 
that (C^'", F'''") is a Markov process whose generator is the closure of the 
operator 

(6.4) A'^'^f{c,v) = 2nv^J{c,v) +nm°''^[f{c,-v) - f{c,v)], 
acting on ah / GP(i'5'"), where 

V{A'n = (/ G C^''°(^ctt X £;siopc) : §^ . ^ j. 

I oc {o,T^."}xi<;,iop, J 

Furthermore, the analogous argument to that of Lemma 6.2 implies that 
the distribution of the pair same distribution as the 

random evolution which is the unique solution to the system 

(6.5) C^^'^ = n 2V^''^dv, y„^'" = (-l) <5''" , 





where is a unit rate Poisson process. In other words, the rescaled process 
C"''" evolves deterministically with speed 2n, and changes the sign at rate 
times a counting process whose rate is governed by the rescaled catalyst 
mass process 7^*°*'". 

Step 1 (The velocity process). The convergence result stated in The- 
orem 1 relies on the fact that the velocity component y*''" evolves much 
faster than the contour component C^'", as is it clear from (6.5). Hence in 
the limit the velocity process will average out and can be replaced with its 
stationary measure. 

If is the occupation-time measure of y*^'" on -Esiopo that is, for n > 0, 
and V e -Esiopc 

(6.6) F"([0,n] X W) := J\^^^{V^r) dn' , 
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then it is clear from the description (6.5) of F*^'" that 

r" ^ r = A vr, 

n— >oo 

where A denotes Lebesgue measure on [0,cx)), and7r({l}) =7r({-l}) = In 
the hmit the contour component will have spent on average half the time 
increasing and half the time decreasing. 

Step 2 (Averaging for martingale problems). The proof of Theorem 1 
will rely on the following result taken from Theorem 2.1 in [17] and adapted 
to our specific situation in which the state spaces are compact. 

Proposition 6.4 (Stochastic averaging). Suppose there is an operator 
-.ViA^) <ZC°{[0,T^]) ^ C^{[0,T^] X {-1,1}) such that: 

(i) for all f e V{A^) there is a process e^'J'" for which 
(6.7) f /(Cf ") - f\A'f){Cl'\ ds + eF'") 
is a martingale, 

(ii) D(A^) is dense in C'^([0, r^]) with respect to the uniform topology 
and 

(iii) for f E P(A^) and T > there exists p > 1 such that 

rT 



(6.8) supE 

n 

and 



|AV(C'f'",1//'")|P(is 



< oo 



(6.9) lim E 



sup|e^ I 

t<T 



0. 



Then for defined as in (6.6) 
{(C"''",r")}„6N is relatively compact in V{[G,oo) x A^([0,oo) x {-1,1})), 

where A^([0,cxo) x {—1,1}) is the space of all measures on [0,oo) x { — 1,1} 
for which ;u([0,n) x { — 1, 1}) = u, and for any limit point {C,^ ,1^) there exists 
a filtration such that 

V(Cf)-/J E A'f{Ct^^,v)ds7:{v} 



1-G{-1,1} 



t>0 



IS a man 



tingale with respect to this filtration, for all f G ^{A^). 



Step 3 (The averaging- martingale problem). Recall the operator {A^ , 
T>[A^)) from (4.3) and (4.4) which is the generator for the rescaled contour 
process C"^'". Our goal is to show that all three assumptions (i)-(iii) of 
Proposition 6.4 above are satisfied. 
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(i) We first show that we can define smah error processes {e^'-^'"",/ € 
V{A^)} such that 



i>0 



is a martingale for all / G ^{A^). 

Notice that / G P(74'') if and only if there exists a function g G C^([0, t^]) 
with 5(0) = (/(r"^) = such that 

(6.10) f{x) = f{0)+ r Xsg{s)ds 

Jo 

for all X > 0. Let / G V{A^) be of the form (6.10) and let 
r(c) :=/(0)+ 

Apply the operator {A^'"','D{A^''^)) from (6.4) to functions /" given by 



nrjc 



to get 



i^'"r (c,«) = 2nv{n'{c) - 2n2^r'"-^(r)'(c) + 2v' (-L_{p)'\ [c) 

nr]c \r]c / 



Let then for all t>0, 

+ fiA^f - i'5'"/")((5f'", y/'") ds. 
Jo 

Since P{Cf'', V"/'") -/o i^'"/"(C'f'", t^/'") is a martingale for all P G P(i'^'"), 
it follows that (6.7) holds for all / G V{A^). 

(ii) We next show that the domain D(yl^) is dense in the space of con- 
tinuous functions on [0, . 

Lemma 6.5 (Dense domain). Fix (5 > and X G C°([0,r'^]). Then the 
set of functions J- defined by 

^ := {/ : /(c) =C + l'x,,g{c') dc'; C G M, g G C\[0, A),g\{o,r^} = o| 

is dense in C°([0,r^]). 
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Proof. It is well known that each continuous function on [0,r ] can be 
approximated by piecewise linear functions. It is therefore enough to show 
that any piecewise linear function can be approximated by functions in T . 
This follows by continuity of X and the fact that X„ > (5, for all u S [0,r^]. 
□ 

(iii) We finally verify the last point. It is standard to show that (6.8) holds 
for all / € T){^Al \ T > and p > 1. Moreover, since is bounded 

by |, for aU t > 0, /" ^ /, and - A^f\\ < - T"^'"! ||c/'|| + |1 - 

^"s^lllff"!! — ^ 0, (6.9) is satisfied as well. 

■J" n—*oo 

Altogether we can apply Proposition 6.4 to the effect that the family of 
rescaled contours {C^'"';n G N} is relatively compact in law and any limit 
point satisfies the (A^, 'D(A^))-martingale problem. 

Step 4 (Uniqueness of the limit-martingale problem). We next show 
that the (A'',I?(^^))-martingale problem has a unique solution , for which 
we first need the following lemma which characterizes solutions of transforms 
of a reflecting Brownian motion. Recall that X is a Feller diffusion started at 

Xo = 1 and is the first time it falls below 5. Let s : [0,t^] — > [0,/J" Xudu] 
defined by 

(6.11) s{x) := r Xudu, 

Jo 

denote the scale function and denote its inverse by s~^. 
Consider the operator 

{B\V{B')) := (2X,.i(.)r, {h G 0^+ [0, oo) : /i'||o,s(r^)} = 0}). 



Lemma 6.6 (Relation to Brownian motion). Let 6 >0. If solves the 
{A^ ,7) [A^)) -martingale problem, then 

Bt:=s{Ct), tG[0,oo), 
solves the {B^ ,'D{B^))-martingale problem. 

Proof. Fix H G C^^ ^^^g^^ [0, oo) with i7'||Q ,,(^«-)| = 0. It is easy to check 

that then HosGViA^). 

We therefore obtain that for all H £ [0, oo) such that -ffl^Q .,(-^5)} = 
the process 



t>o 



t>o 



{Hos){Cf)- A'{Hos){Ci)du 

H{Bt)- j\x,-.^B^)H"{Bu)du 
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is a martingale. Here we have used that A^{H o s){C) = 2{H' o s)'(C) = 
2X^,{H" os){C). □ 

Proposition 6.7 (Uniqueness). The [A^ ,V{A^)) -martingale problem 
has a unique solution C,^ . 

Proof of Proposition 6.7. Assume that C'''^ and C^'^ are two solu- 
tions of the (^^,I?(A''))-martingale problem. By Theorem 5.6 in [26], since 
{Xu)u>f) is bounded away from zero, the (i?'', D(i?^))-martingale problem 
has a unique solution. Hence 

/ Xu du= Xu du. 

Since the scale function is strictly increasing on [0,r^], the one-dimensional 
distributions of C,^'^ and C,^''^ agree. It follows from Theorem 4.4.2 in [12] that 

therefore C^'^ = C'''^- □ 

Step 5 (Conclusion). We close the section by giving a proof of Theorem 

1. 



Proof of Theorem 1. We have shown in Step 2 that the sequence 
(C"^'").„gPj is relatively compact, and that any limit point C,^ of C"^'" is a 
solution of the (^'^, 'D(A'^))-martingale problem. This proves existence of a 
solution for all (5 > 0. 

Furthermore, by Proposition 6.7, this martingale problem has only one 
solution. That is, the (yl'', D(A'^))-martingale problem is well posed and if 
C,^ is its unique solution (4.5) holds. 

Finally, it follows from Theorem 4.4.2 in [12] that C,^ has the Markov 
property. □ 

7. Proof of convergence of the rescaled reactant forests (Theorem 2). 

In this section we verify that the sequence G N} converges in law. 

So far we have shown in Section 5 that this sequence of laws is relatively 
compact. It therefore remains to show that there is a unique limit law. We 
first give a proof of Corollary 4.2. 

Proof of Corollary 4.2. Fix 5 > 0. By Proposition 4.1 there exists 
a random forest Y^™ such that along a subsequence {n}^)j^^^, 

(|for,nfe. ^ (yfor. ^) 

and hence, by the joint continuity of the truncation map Q. (•) : x T''°°'' — > 
-jproot^ we have 

(Qt^,n. (f-'"'=);r?"'=) =^ {QAY'°');X). 

fe— »oo 
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On the other hand we have proven in Theorem 1 that 

k—foo 

Finahy, since the map T : (7^+ [0, oo) — > T''°°'' is continuous, and for ah 
^for g jroot ijg^^g T{C{C^°';a)) = we know that for ah 5>0, 

(Qt,.,(f°'-'"'=);fr'=)=^(r(C^);X), 

K— >00 

which proves (4.6). □ 

Proof of Theorem 2. It follows from Corohary 4.2 that for aU limit 
forests of {ff°'''";n G N}, {Q^s{Y^'"'); X) equals in distribution {T{C^);X). 
We win use this to show that the family {£[(Q^i(yf°'');X)];5 > 0} is a 
Cauchy sequence in the Prohorov metric of probability measures. For that 
fix < (5' < (5, and recall that the Prohorov metric satisfies (see, e.g.. Theo- 
rem 3.1.2 in [12]), 

dpMTiC'y,X),CiTiC''y,X)) 

:= inf inf{e > : ^{((T, p), {T' , p')) : d^^.oo. ((T, p), (T', p')) >e]< e], 

where the infimum is taken over all probability measures p on T''°°* x T''°°* 
with marginals C(T{C^);X) and C(T{C^');X). 

A particular choice of p is the following: pick an excursion (^^ according 
to C{T{C^ )), and recall that l'^^{C,^ ) is the local-time process of C,^ at level 
Tj. Write C,^' '^'^ for Q^' time changed by the inverse of 1 1-^ /q l{Cf' ^ 7"^} ds. 
That is, C^''^^ is C^' with the sub -excursions above level excised and the 
gaps closed up. Then T(C^''J-'^ ) equals in law T(C^) and clearly, 

doH-(T(C^'),T(C^''i-^)) = dH(T(C^'),T(C^''i-^)) = r^' 
Hence, 

dpr(£(r(C'^);X),£(r(C'');X)) < T^' 

which proves that {C\{Q^s{Y);X)\;8 > 0} is Cauchy, as claimed. 

Since (T™"* , dQjjroot ) is complete, the space of probability measures on 
^IToot ^ (^^^^^Qt ) ig complete as well, and we can therefore find a P G i (T''°°* ) 
such that C{Q (T)) j=^^ , and Theorem 2 is proved. □ 

8. Proofs for the reactant limit point process (Theorem 3). In this sec- 
tion we derive the distribution of the reactant limit point process by taking 
a limit of the point processes for the finite reactant forests, as well as con- 
structing the limit point process from the limit contour processes . We 
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start by showing that the point processes of the rescaled truncated reactant 
forests converge. We then show that the hmit of the point processes for the 
truncated reactant is equal in distribution to the point processes associated 
with the contours solving the (^"^j P(74''))-martingale problem. 

Throughout we assume that {fj^°^'"';n G N} and X are given on one prob- 
ability space such that (4.1) holds. Let 

:=inf{i>0:er = 0}. 

The first lemma describes the distribution of the reactant particle point 
process (prior to any rescaling). Recall that 7/*°* equals in distribution 
by definition. 

Lemma 8.1 (Reactant point processes prior to rescaling). For all t G 
{0,R^'^), the reactant point process H* is a simple point process {(i,Tj);i = 
1, . . . , — 1}, where given the {rj; i = 1, . . . , — 1} are independent 
and identically distributed with 

{^ + ih^s dsY JoVf^ds 

Proof. By Lemma 6.1, the pair {C,V) has the strong Markov prop- 
erty. Hence the excursions below t of C are independent and identically 
distributed. 

Notice that an excursion below t equals in distribution the contour process 
of a reactant tree run from time t backward, that is, an upside down tree 
whose branching rates are governed by the process (?7j:°*^)o<s<t, and which is 
conditioned to have height less than t. The maximal depth of the excursion 
is equivalent in distribution to the extinction time of the reactant process 
run with branching rates (ry*°*5)o<s<t conditioned to go extinct by time t. 

For a critical binary branching process with time-inhomogeneous rates 
(As)s>o the population size process {Ns)s>o satisfies (see, e.g., [15], XVII. 10. 11) 

(8.1) F{Nt = 0}- ^^^''^^ 



1 + JqXs ds 

With the special choice Ag := r/^-sj S^t by explicit calculation 
P{ti >h}= FiNt.h = 0\Nt = 0) = '^p^^^tTo}^ 

l + llvl°'ds /oSr^s 

Differentiating this expression with respect to h gives our claim. □ 
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We can now proceed with the proof of Theorem 3. 

Proof of Theorem 3. For all n G N, recall that T^'" from (3.1) is the 
first time the rescaled mass process of the catalyst hits 0, and let 

i?°'":=inf{t>0:er'" = 0}. 

We first examine the distribution of the number of trees in a truncated 
reactant forest and the genealogical distances for a fixed n. 

Step 1 (The point process in the nth approximation step). Choose a 
sequence (tn)n-^oo and t> such that tn — > t. Recall that 11* and are the 

71— »00 

genealogical point processes and IT*"'" and H*"'" are their rescaled versions 
from Definitions 3.5 and 3.8. In order to determine the number of trees in 
the forest, notice that given that t„ ^ T^''^ a i?"'" and given the value of 
independence of the line segments stated in Lemma 6.1, we have 
that the number t„ := H*"'"({i, . . . - ^} x {0}) of points at height 

satisfies 

(t„;r?t°*'") ^ Bin(ner " > tn}), 

where f" is defined as the extinction time of with branching rates (n x 
fjt-s)o<s<t started with one individual, and Bin(n,p) denotes a random vari- 
able which is binomially distributed with parameters n and p. 

Fix an arbitrary fc„ G {1, . . . - 1}. Let Kn := H*"'"({i, . . . , ^} x 

{0}) be the random number of points between ^ and ^ at height 0. Since 
we choose the kn first among the n^j"*'" — 1 many excursions we get from 
the i.i.d. structure that 

(K„;r?*°*'")^Bin(A;„,P{f">tJ). 

Furthermore, given the values of (k„; t]*"*'"), 

Bm{kn - (K„;r?t°*'"),P(f" > /i„|f" < t„)). 





{-•■■ 


kn 1 




L n 


' n J 



Step 2 (The limit point process). We now evaluate the asymptotics 
as n — 5- oo. Pick a sequence {kn)nGn on N such that — — > uYt^ for some 

u £ [0,1]. Since (8.1) implies, given the catalyst total mass processes fy*°*'° 
and X, that for (hn) E [0,t] such that hn^h for some h G [0,t], 

nP{f">/i„} = . " ^ >(f X,ds^ 

l + n //;_;^^ fff ' ds \Jt-h 
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it follows, by the usual Poisson approximation of the Binomial distribution, 
that there exists a point process such that 

(8.3) (k„; „=^(«*([0, uYt] X {0}); X) ^ Pois . 



Next fix /i G (0,t), and let (/i„) be a sequence in (0,t„) such that hn — > 
/i. Note that the first argument of the right-hand side in (8.2) is by (8.3) 
asymptotically equivalent to uiiYt. The second argument is of order n~^. 
Namely by Lemma 8.1, 

nP{f">/i„|f"<t„} 

nP{T" > hn} - nP{f " > tn} 



(1 + ^ (1 + ^ /o" ds) n J^- vT'- ds 

1 1 



Hence, once more using the Poisson approximation, 











t n 


' n J 



(8.4) =»(H*([0,nyj] X (0,t-/i]);X) 



= Pois ( uYt 



1 1 



V \J^_f^Xsds jQXsds 

Since the limiting intensity measure is absolutely continuous with respect 
to the Lebesgue measure, (8.4) implies the weak convergence of the point 
processes, and hence Theorem 3 is proved. □ 



We next show that the reactant limit point process can be derived from 
the unique solutions of the (A'',I?(A''))-martingale problem. We begin with 
the proof of part (a), that is, the existence of a local-time process related to 



Proof of Proposition 4.3. (a) For that purpose we rely on the stan- 
dard characterization (see, e.g.. Section VII. 3 in [25]) of a one-dimensional 
diffusion by its scale function s : [0,oo) — > M and speed measure m, which is 
based on its construction from a Brownian motion. 

Suppose C is an M-valued diffusion with scale function s and speed mea- 
sure m, such that s is differentiable and m is absolutely continuous with 
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respect to the Lebesgue measure. Let m{dx) =m'{x)dx. Then the genera- 
tor of the diffusion C, is given by 

with domain given by 

VIA) ■={f:f = C+ I s'(x)g(x)dx, where ^ G C0([0, oo))l . 
[Jo m' ) 

Then the process B := {Bu)u>o defined by Bu ■= s{Cu),u > 0, is a diffusion 
whose scale function is the identity. Recall that since the scale function is 
nondecreasing it has a well-defined inverse 

s~^{y) := inf{x : s{x) > y}. 

Then the speed measure uib and the diffusion coefficient o"^ of B are 

(compare also with Lemma 6.6). 

Let /3 := (/3u)n>o be a standard Brownian motion with local-time process 
£*(/3) at level t and 7 be the random-time change defined by 

Jo s'{s '-[Pv)) 

Martingale theory implies that the function 7 is such that for its inverse 
7~"^(?;) := {u : jiu) > v} we have 

iBu)u>0 = (/57-1(m))„>o- 

Consequently, we have a representation of the diffusion C by a distribu- 
tionally equivalent process 

(8.7) (C.).>o = (s-'(/37-i(n)))„>o- □ 

We next derive a relation between the local times of P and C- 



Lemma 8.2 (Excursion process of a diffusion). Let jS he the standard 
Brownian motion, and Q he the diffusion with scale function s and speed 
measure density m' . Let i^{C) the local-time process of C at level t. Let 
denote the downward excursions of ( from level t indexed hy the local time 
^*((^), and let tt^'^ := {(^, inf(e^))} he the point process of their depths. 
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(i) The local-time processes £*(C) := (^m(C))m>0 of the diffusion at the 
level t>0 and ^*(/3) := (^u(/?))u>o of Brownian motion j3 at level t>0 satisfy 

(ii) The point process vr^'* has the following law: the marks on [0, 1] x {t} 
separating contributions from distinct excursions of C, above that reach level 
t are points of a Poisson process with rate 

s'{t){s{t)-sm-\ 

Each such excursion path of Q above contributes a set of downward excur- 
sions ej of C below level t whose depths are points of a Poisson point process 
with intensity measure 

i<'^'^ = di0n^'\dh) 

and 



Proof, (i) By definition, 

tie) = Jim — l{^^e{t-s,t+s)} d{C, Ov 

= l^lhl d{B,B)^ 
s'{t) ^ 

Recall that 7 is the random-time change from (8.6). We have 
' Jo s'{s-^{(],)) J '^^^> s'{s-^t)) 

Hence, 



1 

(B) = lim — l{B,^^t-e,t+e)} d{B, B)y 



1 r-^("). m'{s-HM) s'{s-H(3,)) 



and consequently 

(8-9) tiO = ^f,%^W)- 
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(ii) By Ito's excursion theory vr^'* is Poisson with an intensity measure 
(see, e.g., [24], Vol. 2, VII.47) 

H?'* = de(g> n'^'\mf{ej) G d{t - h)) 

for some cr-finite measure n^'* on [0,i]. Every excursion ej of C below t 
corresponds to a time changed excursion of (3 below s{t). Since the number 
of such excursions in the two processes is related by (8.9), we have 

n«'*{inf(e7) >t-h} = s'(t)n^'^W{inf(e7) > s{t) - s{h)} 

= s'{t){s{t)-s{h)r\ 

Now, each excursion of C, above that reaches height t has local time at 
level t equal to o(0 where rj^o = inf{?/ > : Cn = 0, if Co = Excursion 
theory also gives that the local time of a Markov process at level t run 
until the first hitting time of level is distributed as an exponential variable 
with parameter n^'*{sup(e^) >i} = s'{t){s{t) — s(0))~^. Hence the marks on 
level t separating the contribution to the point process vr'''* from each such 
distinct excursion form a Poisson point process on [0, 1] x {t} with this rate. 
□ 



We will next use Lemma 8.2 to give the proof of Proposition 4.3(b). 

Proof of Proposition 4.3. (b) Fix 5 > 0, and let C,^ be the solution 
of the (A'', P(^^)) -martingale problem. 

Let tt'' '* be the point process of depths of downward excursions of C,^ 
below some fixed level t, indexed by the local-time process l\C,^). Recah 
from (6.11) and (8.5) that s is the scale function and m is the speed measure. 
By (8.8) with (6.11) vr'' '* is Poisson with intensity measure 

Xhdh 



H« '\du ® dh) = dt® n^'* = WiiC^)] ® Xt 



The marks separating contribution from distinct excursions reaching level i, 
in other words from different trees, form a Poisson point process on [0, 1] x 
{t] with rate XtU^X^ds)-'^ . 

Let u* := inf{u > 0:£|J(2|/3|) = 1} be the first time the local time of the 
standard reflected Brownian motion 2|/3| at level reaches 1. 

Using Lemma 8.2 we have for all t <t^ , 

{eU\C'\)Xt)o<t<rS = (<-l(..)(2|/3|))o<t<.- 

Recall that the second Ray-Knight theorem implies that the local-time pro- 
cess ili^* (2|/3|) as a function of the level t until the fixed time u* is distributed 
as a Feller diffusion, that is, 

{ei,{2\p\))t>o = {Zt)t>o, 
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where Z satisfies the first part of (3.3), that is, 

dZt = dW^ with Zo = 1 . 
If 7* :=7(ti*), then (8.9) implies that 

(£*.(|C^|)X,)o<,<.. ^ (tf ^(2|/3|))o<i<.. = (>^*)o<t<.^, 

where Y satisfies the second part of (3.3) with another independent set of 
Brownian motions, that is, 

dYt = \js'{t)Yt dWj = ^fZ^Yt dWj with = 1 • 

Hence the local-time process of C,^ at level t equals in distribution (li)t>o 
from (3.3). The claim of Proposition 4.3 follows. □ 

9. Proof for the joint law (Theorem 4). In this section we show the 
convergence for the joint law of the catalyst and reactant forests. 

Proof of Theorem 4. In order to show (4.8) it is enough to show the 
joint convergence of the pair consisting of the catalyst total mass process 
and the reactant forest, that is, 

(9.1) (77*°*'",^^°"^'") ^ (x,yf°n. 

n — >oo 

This follows since given the values of the catalyst mass process the reactant 
forest and the catalyst forest are independent. 

So far we have shown that 71*°*'" =^ X and that under assumption (4.1), 

n— ►00 

l-^for.n. ^tot,n\ _^ (y*'™;X). Heucc, in Order to show (9.1), it suffices to show 

n — >oo 

that the law of the reactant forest depends continuously on the catalyst 
mass path. For that purpose recall from Theorem 2 that for all 5 > 0, 

{Q&iJ^''^)\X) = {T{C^);X). Since the map T : (7°+ [0, 00) : ^ T"-""* is con- 
tinuous, the claim follows by observing from (8.7) that (^^ is continuous in 
X := {Xt)t(z{o,TO)- n 

10. Proof of differences between reactant and "classical" forest. In this 
section we prove the results on the comparison of the reactant limit forest 
with the classical forest. 

Proof of Proposition 4.4. We have specified the distribution of both 
2'^°'' and Y^"^ using collections of point processes vr^'* and vr^'*. For each tree 
in the forest Y^°'^ the distribution of vr^'* is given by the intensity measure 
^C* from (4.7), and for a standard Brownian tree in Z^°^ distribution of vr^'* 
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is given by the intensity measure in (3.8). It is easy to check that for 
each t £ (0,r°], 

(10.1) ((st(H^'*);X))t>o = ((H'''*))t>o, 

where St apphes to the second coordinate of the bivariate measure bmce 
the genealogical point process records the distances between points at the 
same level in the forest, (10.1) implies the claim. □ 

Proof of Proposition 4.5. 



E 



(aQt(yf°0)^ 



\ui r dU2fW{[uiYt,U2Yt] X {0}) /0} 
JO 











1 - 


/ dui 


f dU2E 


exp<j 




Jo 


Jo 












1 - 


/ dui 


f dU2E 


exp| 




Jo 


Jo 





Yt\ui - U2\ 

lo ds 

t 



Xg ds 



\Ui - U2\ 
l + \ui- U2\ 



where we have used that a branching diffusion Yt with time-dependent 
branching rates h{t) satisfies 



yo = y\= exp 



1 + A /q h{u) du 



By assumption E[(/q dsX^) ^] = ^ . Therefore, Jensen inequality yields 

rt \ -1 

exp — I / Xf,ds 



E 



\Ui - U2\ 



>exp-<^E 



l + \ui- U2 



Xg ds 



Ul - U2\ 



1 + Ml - U2\ 



■ exp 



Z \Ui — U2\ 

t l + |ni — n2| 



Hence 



E 



{fi''^)^\d{u, u'))l{dYfor {U, u') = 2t} 



<1 — J dui I du2 exp"! — — 

= E 



\Ui - U2\ 



(aQt(zf°o)2 



t 1 + - U2 

(/i*'^)®2(d(u, u'))l{dzfo. (n, u') = 2t} 



□ 
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We prepare the proof of Corollary 4.7 with the following lemma. Assume 
that we are given an excursion e with L{e) = inf{i > : e(t) = 0} < oo. Recall 
from (2.1) the map T which sends e to a rooted ordered forest T(e) = 
(Te, dy^, [0], <Te) associated with e. 

Lemma 10.1. The l"^ -length of a forest T(e) associated with an excur- 
sion e e C°+ [0, oo) with L(e) = inf {t > : e(t) = 0} < oo equals the quadratic 
variation of e 

(10.2) Xe:=P-lhn sup I |e(ti+i) - e(t0l4 • 

^^^0<tQ<ti< -<tn<L{e), [i^i ) 
max{|ti+i-ti|<e} 

Proof. Assume that we are given an excursion e with L{e) = inf{t > 
: e{t) = 0} < oo. Assume that Te has a well-defined 1^-length. Since edges of 
positive length do not contribute to the F-lengths of Tg, (4.9) holds even if we 
further restrict the finite subsets of Tg to those with inf{e > : (T^^Y = 
j^branchj _> Q, as n ^ OO. Here T^f^^'ich cigi;iotes the set of all inner branch 
points of Te, where x gT^ is an inner branch point if Tg \ {x} decomposes Tg 
into more than 2 connected components. Likewise, monotone regions in the 
excursion e do not contribute to the quadratic variation of e, so in (10.2) 
we may restrict the set of time points to only the finite subsets of the set of 
times < t™" < • • • < tf ™ < • • • < L(e) such that e(t™"^) is a local minimum. 
Since e is continuous it can have only a countable number of local minima 
on an interval [0,L(e)]. Now we observe that the local minima of e precisely 
correspond to the branch points of Tg , so 

Xe = P- lim sup |e(C;?) - e(tr)|n 

= P- hm sup + e(tr ) - 2 min e(. 



i+i 

2 



i+iJ 



n— »oo 

a,beT' 



= l'{T{e)). □ 

Proof of Corollary 4.7. Lemma 10.1 states in particular that if ( is 
assumed to be an excursion associated with the reactant forest then = oo 
by Lemma 4.6. Hence C cannot be a diffusion path and the claim follows. 
□ 

Proof of Lemma 4.6. By its definition (4.9), {"^{Z^"^) equals in distri- 
bution the quadratic variation of a reflected Brownian motion 2|/3|./\£-i(]^) 
which is known to be £(2|/3|)~^(1) < oo, almost surely. 
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As for the reactant limit tree, on the event {p^ < t^}, 1^(1^'^°'') equals 
in distribution for suitably small positive 5 the quadratic variation of the 
well-posed solution (^^ of the (A*^, P(74'^))-martingale problem stopped at the 
first time the local time of at reaches 1 This quadratic 

variation is finite almost surely and remains finite as 6^0. 

On the other hand, on {p^ > t^} we show that the quadratic variation of 
the contours diverges as 5 ^ 0. In fact, by (10.2), 

f{Q^s(Y^°';X)) = (C^,C'^)(£0(^i))-i(i). 

Recall from (8.7) that, given X, the law of can be expressed in terms of 
a function of a time changed Brownian motion /3: 

(C«)«>o = is~^{Bu))u>o, iBu)u>o = (/37-i(«))«>o- 
Since ^{u) = Jq )) we have d{B,B)u = s'{s~^{Bu)) du, and 

d{C^, C\ = ( ' d{B, B)u = du = ^ du. 



dt 



s.s'{s-HBu))J ' ' s'{s~\Bu)) 
Hence, by the occupation time formula 

rnc'r^W 1 , fsir^) ^(/3);(,o(^)-i(i)) 



JO Jo 



Xy 



by changing variables. We next note that inf{f > : £(/3)^^^^o(^)-i(i)) = 0} 
p^ > T^, so we will have c = inf^gjo.r"] ^(f^yj{fi{p)-'^{i)) dv > and 



l\QAY''";X))>c r -^dv 
Jo y^v 



— > oo 

5^0 



using the known fact that Jq dv = oo (see, e.g.. Lemma 3.1 in [5]). □ 

Acknowledgment. The authors thank the referee for extremely careful 
reading of the paper. Her or his suggestions and comments have led to many 
improvements in the paper. 

REFERENCES 

[1] Aldous, D. (1991). The continuum random tree. I. Ann. Probab. 19 1-28. 
MR1085326 

[2] Aldous, D. (1991). The continuum random tree. II. An overview. In Stochastic 
Analysis (Durham, 1990). London Mathematical Society Lecture Note Series 
167 23-70. Cambridge Univ. Press, Cambridge. MR1166406 



GENEALOGY OF CATALYTIG BRANGHING MODELS 



41 



[3] Aldous, D. (1993). The continuum random tree. IIL Ann. Probab. 21 248-289. 
MR1207226 

[4] Aldous, D. J. (1999). Deterministic and stochastic models for coalescence (aggre- 
gation and coagulation): A review of the mean-field theory for probabilists. 
Bernoulli 5 3-48. MR1673235 

[5] Athreya, S. and Winter, A. (2005). Spatial coupling of neutral measure- valued 
population models. Stochastic Process. Appl. 115 891-906. MR2134483 

[6] Ghiswell, I. (2001). Introduction to A-Trees. World Scientific, River Edge, NJ. 
MR1851337 

[7] Donnelly, P. and Kurtz, T. G. (1999). Particle representations for measure-valued 

population models. Ann. Probab. 27 166-205. MR1681126 
[8] Dress, A. W. M. (1984). Trees, tight extensions of metric spaces, and the coho- 

mological dimension of certain groups: A note on combinatorial properties of 

metric spaces. Adv. m Math. 53 321-402. MR753872 
[9] Dress, A. W. M., Moulton, V. and Terhalle, W. F. (1996). T-theorie. Europ. 

J. Combinatorics 17 161-175. 
[10] Dress, A. W. M. and Terhalle, W. F. (1996). The real tree. Adv. in Math. 120 

283-301. MR1397084 

[11] DuQUESNE, T. and Le Gall, J.-F. (2002). Random trees. Levy processes and spatial 

branching processes. Asterisque 281 vi-147. MR1954248 
[12] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and 

Convergence. Wiley, New York. MR838085 
[13] Evans, S. N., Pitman, J. and Winter, A. (2006). Rayleigh processes, real trees, 

and root growth with re-grafting. Probab. Theory Related Fields 134 81-126. 

MR2221786 

[14] Evans, S. N. and Winter, A. (2006). Subtree prune and re-graft: A reversible real- 
tree valued Markov chain. Ann. Probab. To appear. 34 918-961. 

[15] Feller, W. (1968). An Introduction to Probability Theory and Its Applications. Vol. 
I, 3rd ed. Wiley, New York. MR0228020 

[16] Greven, a., Klenke, A. and Wakolbinger, A. (1999). The longtime behavior 
of branching random walk in a catalytic medium. Electron. J. Probab. 4 80. 
MR1690316 

[17] Kurtz, T. G. (1992). Averaging for martingale problems and stochastic approxima- 
tion. In Applied Stochastic Analysis (New Brunswick, NJ, 1991). Lecture Notes 
m Control and Information Sciences 177 186-209. Springer, Berlin. MR1169928 

[18] Lamperti, J. (1967). The limit of a sequence of branching processes. Z. Wahrsch. 
Verw. Gebiete 7 271-288. MR0217893 

[19] Le Gall, ,J.-F. (1999). Spatial Branching Processes, Random Snakes and Partial 
Differential Equations. Birkhauser, Basel. MR1714707 

[20] Penssel, C. (2003). Interacting Feller diffusions in catalytic media. Ph.D. thesis. 
Institute of Math., Erlangen, Germany. 

[21] Pitman, J. (2006). Combinatorial Stochastic Processes. Lecture Notes in Mathemat- 
ics 1875. Springer, Berlin. MR2245368 

[22] POPOVIC, L. (2004). Asymptotic genealogy of a critical branching process. Ann. Appl. 
Probab. 14 2120-2148. MR2100386 

[23] Protter, p. E. (1977). On the existence, uniqueness, convergence and explosions 
of solutions of systems of stochastic integral equations. Ann. Probab. 5 243-261. 
MR0431380 



42 



A. GREVEN, L. POPOVIC AND A. WINTER 



[24] Rogers, L. C. G. and Williams, D. (1987). Diffusions, Markov Processes, and Mar- 
tingales: ltd Calculus. Wiley Series in Probability and Mathematical Statistics: 
Probability and Mathematical Statistics. 2. Wiley, New York. MR921238 

[25] Revuz, D. and YOR, M. (1999). Continuous Martingales and Brownian Motion, 3rd 
ed. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles 
of Mathematical Sciences] 293. Springer, Berlin. MR1725357 

[26] Stroock, D. W. and Varadhan, S. R. S. (1969). Diffusion processes with contin- 
uous coefficients. I. Comm. Pure Appl. Math. 22 345-400. MR0253426 

[27] Terhalle, W. F. (1997). _R-trees and symmetric differences of sets. European J. 
Combm. 18 825-833. MR1478827 



A. Greven 
A. Winter 

Mathematisches Institut 
University Erlangen-Nurnberg 

BiSMARCKSTR. 11/2 

91054 Erlangen 
Germany 

E-MAIL: greven@mi.uni-erlangen.de 
winter@mi.uni-erlangen.de 



L. PoPOViC 

Department of Mathematics and Statistics 
Concordia University 
Montreal QC H3G 1M8 
Canada 

E-MAIL: lpopovic@mathstat. Concordia. ca 



